Papers by M. Hamza Mughal
Enhancing Spoken Discourse Modeling in Language Models Using Gestural Cues (2025.acl-long)
Copied to clipboard
| Challenge: | linguistic research shows that non-verbal cues, such as gestures, play a crucial role in spoken discourse. |
| Approach: | They propose to integrate gestures into language models by embedding human motion sequences into discrete gesture tokens and aligning them with text embeddables. |
| Outcome: | The proposed model improves on spoken discourse, the authors show . the study aims to improve the accuracy of discourse markers and quantifiers . |
Modeling Turn-Taking with Semantically Informed Gestures (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing computational models of turn-taking relied on verbal cues and prosody. |
| Approach: | They propose a framework that integrates text, audio, and gestures to model multimodal turn-taking using semantic annotations. |
| Outcome: | The proposed framework shows that incorporating semantically guided gestures yields consistent performance gains over baselines. |